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Abstract 


Driver drowsiness is one of the major causes of road accidents and it can lead to serious physical injuries, 
loss of human life, damage to property & loss of money. So a reliable driver drowsiness detection system is 
needed to be implemented, which would alert the driver before anything undesired happens. In this paper, 
design and implementation of *.Driver Drowsiness Detection System with Audio-Visual Warning’ will be 
discussed. This system is to be evolved for car driver, but the scope of this system is far more than it. It can 
be used in any situation where a person’s drowsiness is needed to be monitored. The planned system can use 
a camera that takes pictures of driver’s face and monitors the driver’s eyes in order to detect drowsiness of 
driver. 

Once fatigue is detected, the alarm will be used to alert the driver. The proposed system will work in 
3 main stages, in 1 st stage the face of the driver is detected and tracked. In the 2 nd stage the facial features 
are extracted for further processing. In last stage, eye’s status is monitored. In 3 rd stage it is determined that 
whether the eyes are closed or open. On the basis of this result the warning is issued to the driver. For this 
Raspberry pi with raspbian (Linux) OS is used. The camera will be connected through USB port of 
Raspberry pi. The picture processing will be done using OpenCV. 

KEYWORDS : Advanced Vehicle Safety, Driver Drowsiness Detection, Driver Fatigue, Raspberry-pi, Raspbian, Vehicle Accident 
Warning 

INTRODUCTION 

Drowsiness will be a process in which one level of awareness is reduced because of lacking of sleep or fatigue 
and it can cause the driver constitute sleep quietly. Once the driver is affected by drowsiness driver loses the control of 
the car thus driver could be suddenly deviated from the road and hit an obstacle or a car to overturn. According to 
available statistical data, over 1.3 million people die annually on the road and twenty to fifty million people suffer non 
fatal injuries because of road accidents [1]. Based on police reports, the U.S. National Highway Traffic Safety 
Administration (NHTSA) cautiously estimated that an entire of 100,000 vehicle crashes annually are the direct result of 
driver drowsiness. These crashes resulted in approximately 1,550 deaths, 71,000 injuries and $12.5 billion in monetary 
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losses. In the year 2009, the U.S. National Sleep Foundation (NSF) according that 54% of adult drivers have driven a 
vehicle while feeling drowsy and 28% of them actually fell asleep. There are 2 strategies for drowsiness detection. The 
first one is intrusive strategies and also the other is non-intrusive methods. The intrusive strategies include measurement 
of heartbeat rate, mind wave observation etc. It is most accurate, however it is not realistic, because sensing electrodes 
should have to be connected directly onto the user’s body, and thus it might be annoying and distracting the user. While 
the non-intrusive strategies include the yawn detection, eye closure, eye blinking rate, head pose etc... [2]. It is realistic 
because it doesn't irritate the user while driving because no sensing electrodes would be connected to user’s body. 
Specific system uses raspberry pi which is small in size, less power requirement and has low price compare to other 
computers like desktop and laptop computer solves some issues of existing systems. Face and eye detection is done using 
Haar cascade classifier. Open CV is used to increase efficiency. 

BACKGROUND 

The Raspberry Pi is a basic embedded system having a credit card-sized single board computers developed in 
the UK by the Raspberry Pi Foundation. It is supported on ARM processor, which is used in most smart phones and 
tablets. The raspberry pi is intended to be used as headless (Just a CPU), but is widely used with totally different 
Displays, Touchscreens and multimedia system components. This planned system uses raspberry pi 2 model B, which is 
latest at this time. The physical view of raspberry pi 2B is as shown in figure 1. The raspberry Pi board is likely just a 
CPU which has all necessary components on board for processing. The components on board are as shown in figure 1. It 
has 1 GB RAM, 40 GPIO for connecting inputs/outputs. The CPU is Quad core BCM2836, that runs at 900MHz 
frequency [3]. The processor is based on ARM v7 design. 
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Figure 1: Raspberry Pi 2B [2] 
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Figure 2: Block Diagram of Planned system 

PROPOSED METHODOLOGY 

Overview of the Planned Work 

There are many signs by which we can detect the drowsiness of drivers. They are as given below: 

• Driver may be yawn frequently. 

• Driver is unable to keep eyes open. 

• Driver catches him nodding off and has trouble keeping head up. 

Driver drowsiness detection system is a system that's implemented using picture processing to detect drowsiness 
of the driver. This application are offen terribly helpful to reduce the accidents, because most accidents happens because 
of drowsiness of drivers. With use of this application the driver’s status can be monitored, like yawning, fatigue, closure 
of eyes etc. The status of driver’s face is continuously monitored using the camera. The alarm would be used to give an 
alert, if any sign of drowsiness is detected. 

The planned system can work in 3 main stages: 

1. In 1 st stage the face of the driver is detected and tracked. 

2. In the 2 nd stage the facial expressions are extracted for further processing. 

3. In last stage, eye’s status is monitored. During this last stage it is determined that whether the eyes 

are closed or open. On the idea of this result the warning is issued to the driver. 

For this system I’m going to use Raspberr Pi with raspbian (linux) OS. The camera will be connected through 
USB port of raspberry pi. The picture processing will be done using OpenCV. 

Flow Chart of Planned System 

The flowchart of picture processing part is shown as fig. 3. The camera is connected on Raspberry Pi using USB 
port. The camera needs to be initialized at the start-up of code. The modules of camera will be loaded and it can ready to 
take pictures. The captured pictures are processed to detect face of the driver. If the face is not detected in the processed 
picture, it will continue to capture picture and process it. If the face is detected in- the frame, it will be processed to 
extract facial expressions. The picture will be process to detect open eyes. If the eyes are open, the code will continue to 
capture & process picture. If the eyes are closed, the system will check first, if the timer is ON or OFF. If the timer is 
OFF, it will be turned ON. And if it is ON, it’ll check if the time is greater than two seconds. It time is less than 2 
seconds, it will return and continue to capture and process picture. If the timer exceeds time of two seconds, the alarm 
will be turned ON. The alarm will be played using GPIO port of Raspberry Pi [4]. One pin will be initialized as Output 
and hardware alarm will be connected on that. Pin will be set ON to turn on the alarm. 

After turning ON the alarm, the General Purpose Input Output module of raspberry Pi will be initialized to 
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require input from user. One button will be connected on GPIO module to take input. The alarm can stopped if the button 
is pressed. By this mechanism the driver can turn the alarm off when he/she is awakening, and additionally the system 
will be return to capture and process the pictures. Flow chart shown in figure 3. 



Figure 3: Flowchart of Planned System 
Performance Evaluation 

Speed : 

The speed of system is outlined as the frequency of total program instructions for one explicit cycles. Higher the 
frequency, better the performance. 

Speed =cycles per second 
Speed =1/Tm 

Where, Tm is time taken by one complete cycle of total program instructions. 

Accuracy : 

The accuracy is defined by the ratio of correctly detected eye state, by the overall samples taken by camera. One 
particular cycle can be explained as one sample. 

Accuracy = Nc/Nt 
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Where Nc = number of samples in each and every states 

Nt= total number of taken samples 

Speed = cycles per second 

Speed = 1 / Tm 

Speed = cycles per second 

Speed = 1 / T 

WORK RESULTS 

Face and Eye Detection in Picture 

Following figs. 4(a) & 4(b) shows the detected face and eye from the picture. The face and eye detection is done 
using Haar Cascade Classifier technique. The blue square will be drawn in result picture automatically when the 
execution of python code. 



Figure 4: (a) Detected Face Picture 



Figure 4(b): Detected Eyes in the Picture 
Mouth and Nose Detection in Picture 

Mouth and Nose detection is shown in figs 5(a) & 5(b). The aim of mouth detection is to detect yawning if 
possible for future scope. In the result of nose detection, the nose is detected in face region correctly, but it is also 
detected in background, which is an error. So we can remove this error by limiting the picture search area. First the face 
will be detected, and from that area of face, the other features like eyes, nose & mouth will be detected. This can solve 
the problem. 
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Figure 5(a): Detected Mouth in a Picture 



Figure 5(b): Detected Nose in an Picture 
Face, Eye & Mouth Detection in Picture 


In the figure 6, the detection of Face, Eye & mouth is shown. It’s implemented using the strategy explained in 
previous section [5]. The result is better and accurate this way, because it reduces the possibility of error by limiting the 
search area to only face region. 




Figure 7: Detected Face through webcam 


Face Detection Using Live Webcam 


The detection of face is done using live webcam as shown in fig. 7. During this code, the USB webcam is 
designed for taking pictures and that picture is taken for processing using same methodology as above Face detection is 
accurate using Haar cascade technique compare to other like LBP. 
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Face, Eye, Mouth & Nose Detection using Live Webcam 




Figure 6: Detected Face, Eyes, Mouth and Nose through Webcam 


In the figure above, the detection of Face, Eye, Mouth and Nose using live webcam is shown. It is also using the 
same methodology. This also works in low lighting condition as shown in figure. All the results are accurate for various 
users. 


Open and Closed Eye Detection 



Figure 7: Detected Open and Closed Eyes 


The eye status monitoring is done using Blob detection. Figure shows the detected open and closed eyes. It 
draws circles around eye region as shown in figure. For face and eye detection is done using Haar cascade methods. Only 
eye status monitoring is done using blob detection to check eye status whether it closed or opened. Also it is accurate. 
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Setup of System 



Figure 8: Experimental setup of running system 

The complete hardware configuration is as shown in above figure. There are total 4 LED, White, Yellow, Blue, 
Red are used as shown in above figure. White for power supply. Yellow LED will be on when system start running. Blue 
LED for face detection. If face is detected, then it will be turned on otherwise it will remain off. Red LED is used for eye 
status monitoring. When close eye is detected red LED will be turned on [6]&[7]. If closed eye detected the Red LED 
will be turned off. There is also one switch to stop buzzer. 

Result Analysis 

The accuracy of frontal face and the eyes detection is as shown in figure 14. Face and eye detection is tested 
using Haar cascade algorithm and Local Binary Pattern (LBP) [8]. In graph, we will see that the accuracy of both face 
and eye detection is high using Haar cascade classifier algorithmic rule as compare to the local binary pattern. Accuracy 
of frontal face using Haar cascade and lbp are around 100% and 92% respectively. 

And the accuracy of eye detection using Haar cascade and lbp are around 100% and 79% respectively as shown 
in following figure. 

Accuracy 



Figure 9: Accuracy of Face and Eyes Detection 
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Figure 10: Accuracy and Speed 


CONCLUSIONS 

There are several intrusive and non-intrusive methods to implement driver drowsiness detection system. From 
the study and design of planned work it's clear that usage of raspberry pi and opencv is even a lot of appropriate for this 
specific application in terms of size, cost and power requirement The results are accurate and reliable for detection of 
face, eyes & mouth. The operations are performed on static picture as well on feed of live webcam. It is observed that, 
the result varies due to un-even lighting conditions; however it is accurate even in low light conditions. Specific system 
will be used in real working environment. In future yawning detection and head nodding will be possible. 
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